Skip to content

Add a ZCode host - #218

Open
Stanislav Usoltsev (stanislavus) wants to merge 1 commit into
microsoft:mainfrom
stanislavus:zcode-kimi-hosts
Open

Stanislav Usoltsev (stanislavus) wants to merge 1 commit into
microsoft:mainfrom
stanislavus:zcode-kimi-hosts

Conversation

@stanislavus

Copy link
Copy Markdown

Upstream pull request text: ZCode host for TokenEconomizer

The person pushes the fork branch zcode-kimi-hosts and opens the pull request against
microsoft/TokenEconomizer. Everything under the rule below is the pull request text. The title
is the first line.


Add ZCode as a host (experimental), and document why Kimi Code is not one

What

  • tokeneconomizer/trace/zcode.py: a reader for ZCode's SQLite store
    (<ZCODE_STORAGE_DIR or ~/.zcode>/cli/db/db.sqlite). zcode_export opens it read-only, copies
    one session's tool calls, exit codes and token usage into a JSON file of the plugin's, and the
    other functions read that file: zcode_session_to_claude, zcode_prompt_count,
    zcode_session_usage. The copy has no words: a prompt is counted and not kept, and what the
    model said or reasoned is not read (the same approach as trace/hermes.py).
  • plugin/hosts.py: a ZCode host, and detect support. ZCode's hook payload is Claude-shaped
    (it has session_id, tool_name, tool_input and the rest, beside its own camelCase fields),
    so what tells it apart is its transcript: transcript_path and transcriptPath are one
    temporary zcode-hook-<random>/transcript.jsonl. ZCode also sends the camelCase
    hookEventName that the Grok host takes as its mark, so without this a ZCode payload read as
    Grok's. default_host knows ZCODE_SESSION_ID, after every other agent's variable.
  • plugin/sidecall.py: zcode_call, a side call through zcode -p in a throwaway directory, in
    plan mode (its -p default is yolo), with the acting tools removed by
    --disallowed-tools. It is off until zcode_sidecall is on (or TE_ZCODE_SIDECALL=1), as
    Droid's is.
  • plugin/hooks/zcode-hooks.json: the five hook entries, for a repository's zcode.json.
  • docs/ZCODE.md, a section in docs/PLUGIN.md, README rows, docs/KIMI.md.
  • tests/test_zcode.py, tests/test_kimi.py; tests/conftest.py stubs zcode_call like the
    other side calls.

Why

ZCode (Z.ai's coding agent) runs Claude-shaped hooks and keeps every session in one SQLite file.
The hook's own transcript is a temporary file that holds one line and is deleted when the hook
returns, so no existing reader can see a ZCode session. TokenEconomizer's mechanisms need only
(name, args, result, ok) calls and a prompt count, which the reader gives them.

Two facts of the store are not obvious and are tested: model_usage.input_tokens counts the cached
reads, where the ledger's input does not, so the reader subtracts them; and a subagent's calls are
rows of its own session with query_source = 'subagent', which the parent session paid for, while a
forked session has a parent too and is not added.

Kimi Code

Kimi Code 2.x has no reader here on purpose. The only wire.jsonl records seen are metadata,
config.update and tools.set_active_tools, and whether an exit-0 hook's stdout reaches the model
is not tried. docs/KIMI.md says what to collect, and tests/test_kimi.py has two skipped tests
that name it, and two tests that pin that no adapter claims a Kimi payload.

Tests

python3 -I -m pytest tests experiments/tests

  • tests/test_zcode.py (57 pass and 2 skip, with tests/test_kimi.py): the reader on a synthetic store whose
    JSON keys were read off a real one (no conversation copied); words never reach an export; failure
    is decided by exit code; typed prompts only (reminders, goal continuations and background-task
    notices are not prompts); usage with the cache taken out and the subagent added; the store is
    opened read-only and a missing or foreign store is no session and creates nothing; a session id
    that is no id is refused.
  • Negative cases for detection: a Claude Code payload, a payload with only ZCODE_SESSION_ID in
    its environment, a camelCase-only payload, a Grok payload, a transcript whose two fields differ,
    one with another file name or folder name, no transcript, and a Kimi session path are all not
    ZCode. An unknown --host value is not ZCode.
  • The hooks end to end through hook.main with a payload built as ZCode's own code builds it:
    session recorded, folded once from the store, a second fold finds nothing.
  • Side call: the command (plan mode, empty directory, tools removed, TE_SIDECALL=1, no stdin),
    the off-by-default switch from the variable and from the setting, a reply-less call noted and
    paused.

How it was verified

The full offline suite passes with the change: 1372 passed, 21 skipped in tests and
experiments/tests together, on Python 3.11 with pytest 9.2. The base is the pinned commit 0566b60. The hook code was read from ZCode 0.17.1's bundle, and the store
schema and the JSON keys of its rows from a real 0.17.1 store (keys and aggregate counts only).

Limits

  • Not run under a live ZCode session with the hook file, and zcode -p was not called with a real
    model: what it prints, and whether --disallowed-tools keeps out every tool, are unchecked. That
    is why the side call is off by default. docs/ZCODE.md has the steps to check both.
  • No history import for ZCode yet (the store holds old sessions; nothing reads them).
  • keeps_start_context keeps its default: whether ZCode replays a session start's context on resume
    was not traced.
  • Cache writes are not priced: cache_creation_input_tokens was 0 in every row seen.

@microsoft-github-policy-service

Copy link
Copy Markdown

Stanislav Usoltsev (@stanislavus) please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.

@microsoft-github-policy-service agree [company="{your company}"]

Options:

  • (default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.
@microsoft-github-policy-service agree
  • (when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.
@microsoft-github-policy-service agree company="Microsoft"
Contributor License Agreement

Contribution License Agreement

This Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
and conveys certain license rights to Microsoft Corporation and its affiliates (“Microsoft”) for Your
contributions to Microsoft open source projects. This Agreement is effective as of the latest signature
date below.

  1. Definitions.
    “Code” means the computer software code, whether in human-readable or machine-executable form,
    that is delivered by You to Microsoft under this Agreement.
    “Project” means any of the projects owned or managed by Microsoft and offered under a license
    approved by the Open Source Initiative (www.opensource.org).
    “Submit” is the act of uploading, submitting, transmitting, or distributing code or other content to any
    Project, including but not limited to communication on electronic mailing lists, source code control
    systems, and issue tracking systems that are managed by, or on behalf of, the Project for the purpose of
    discussing and improving that Project, but excluding communication that is conspicuously marked or
    otherwise designated in writing by You as “Not a Submission.”
    “Submission” means the Code and any other copyrightable material Submitted by You, including any
    associated comments and documentation.
  2. Your Submission. You must agree to the terms of this Agreement before making a Submission to any
    Project. This Agreement covers any and all Submissions that You, now or in the future (except as
    described in Section 4 below), Submit to any Project.
  3. Originality of Work. You represent that each of Your Submissions is entirely Your original work.
    Should You wish to Submit materials that are not Your original work, You may Submit them separately
    to the Project if You (a) retain all copyright and license information that was in the materials as You
    received them, (b) in the description accompanying Your Submission, include the phrase “Submission
    containing materials of a third party:” followed by the names of the third party and any licenses or other
    restrictions of which You are aware, and (c) follow any other instructions in the Project’s written
    guidelines concerning Submissions.
  4. Your Employer. References to “employer” in this Agreement include Your employer or anyone else
    for whom You are acting in making Your Submission, e.g. as a contractor, vendor, or agent. If Your
    Submission is made in the course of Your work for an employer or Your employer has intellectual
    property rights in Your Submission by contract or applicable law, You must secure permission from Your
    employer to make the Submission before signing this Agreement. In that case, the term “You” in this
    Agreement will refer to You and the employer collectively. If You change employers in the future and
    desire to Submit additional Submissions for the new employer, then You agree to sign a new Agreement
    and secure permission from the new employer before Submitting those Submissions.
  5. Licenses.
  • Copyright License. You grant Microsoft, and those who receive the Submission directly or
    indirectly from Microsoft, a perpetual, worldwide, non-exclusive, royalty-free, irrevocable license in the
    Submission to reproduce, prepare derivative works of, publicly display, publicly perform, and distribute
    the Submission and such derivative works, and to sublicense any or all of the foregoing rights to third
    parties.
  • Patent License. You grant Microsoft, and those who receive the Submission directly or
    indirectly from Microsoft, a perpetual, worldwide, non-exclusive, royalty-free, irrevocable license under
    Your patent claims that are necessarily infringed by the Submission or the combination of the
    Submission with the Project to which it was Submitted to make, have made, use, offer to sell, sell and
    import or otherwise dispose of the Submission alone or with the Project.
  • Other Rights Reserved. Each party reserves all rights not expressly granted in this Agreement.
    No additional licenses or rights whatsoever (including, without limitation, any implied licenses) are
    granted by implication, exhaustion, estoppel or otherwise.
  1. Representations and Warranties. You represent that You are legally entitled to grant the above
    licenses. You represent that each of Your Submissions is entirely Your original work (except as You may
    have disclosed under Section 3). You represent that You have secured permission from Your employer to
    make the Submission in cases where Your Submission is made in the course of Your work for Your
    employer or Your employer has intellectual property rights in Your Submission by contract or applicable
    law. If You are signing this Agreement on behalf of Your employer, You represent and warrant that You
    have the necessary authority to bind the listed employer to the obligations contained in this Agreement.
    You are not expected to provide support for Your Submission, unless You choose to do so. UNLESS
    REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING, AND EXCEPT FOR THE WARRANTIES
    EXPRESSLY STATED IN SECTIONS 3, 4, AND 6, THE SUBMISSION PROVIDED UNDER THIS AGREEMENT IS
    PROVIDED WITHOUT WARRANTY OF ANY KIND, INCLUDING, BUT NOT LIMITED TO, ANY WARRANTY OF
    NONINFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
  2. Notice to Microsoft. You agree to notify Microsoft in writing of any facts or circumstances of which
    You later become aware that would make Your representations in this Agreement inaccurate in any
    respect.
  3. Information about Submissions. You agree that contributions to Projects and information about
    contributions may be maintained indefinitely and disclosed publicly, including Your name and other
    information that You submit with Your Submission.
  4. Governing Law/Jurisdiction. This Agreement is governed by the laws of the State of Washington, and
    the parties consent to exclusive jurisdiction and venue in the federal courts sitting in King County,
    Washington, unless no federal subject matter jurisdiction exists, in which case the parties consent to
    exclusive jurisdiction and venue in the Superior Court of King County, Washington. The parties waive all
    defenses of lack of personal jurisdiction and forum non-conveniens.
  5. Entire Agreement/Assignment. This Agreement is the entire agreement between the parties, and
    supersedes any and all prior agreements, understandings or communications, written or oral, between
    the parties relating to the subject matter hereof. This Agreement may be assigned by Microsoft.

@aaron-jx-li Aaron Li (aaron-jx-li) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution. This PR needs a few changes before it can be merged:

  1. Kimi Code doesn't belong in this PR. docs/KIMI.md, tests/test_kimi.py, the Kimi text in README.md:193-194 and README.md:513, and docs/PLUGIN.md:751-753 document a host this PR doesn't add, and #234 already rewrites docs/KIMI.md and deletes most of tests/test_kimi.py. Please drop them here so this PR is ZCode only, and let #234 carry the Kimi docs. The Kimi-path negative case in tests/test_zcode.py's detection list can stay. Please also remove the TC-980/TASK-980 references (tests/test_zcode.py:10-11,298) and the instructions block at the top of the PR description.
  2. A subagent's usage is counted twice. docs/ZCODE.md:75 says a subagent's own hook calls carry its own session id, so the plugin records that session and folds it later. zcode_export (tokeneconomizer/trace/zcode.py:144-149) already adds the subagent's query_source = 'subagent' rows to the parent, so folding the subagent's own session counts the same rows again, and its task message counts as a prompt. With make_db from tests/test_zcode.py and one PostToolUse hook for sess_sub, fold_session gives sess_1 input 1500 (including the subagent's 200) and sess_sub input 200 / cache 500 / output 50 / prompts 1. Please keep a subagent's hook calls from creating a session of their own (for example a ZCode.is_subagent that reads ZCode's agent field), or have ZCode.complete fold a session that has a parent_id and only subagent rows as nothing. Add a test where the subagent's hook fires.
  3. Nothing here has run under ZCode. README.md:73 and CLAUDE.md:4 list ZCode with the other agents, but docs/PLUGIN.md:723 says the adapter was "read from its code, not run live". That leaves unchecked the hook file, the $HOME expansion in its command, the trust step and whether additionalContext reaches the model. Every other host in docs/PLUGIN.md §3 has at least one live check. Since you have 0.17.1 installed, please run the "Verify it" steps (docs/ZCODE.md:49-57) and record the results in docs/PLUGIN.md with the version and date: the session listed after its first hook, the fold with its prompts and tokens, and a fact the model repeated. Please also add a ZCode row to "Supported versions" (README.md:287) and a ZCode entry under §5 "Known limitations", as the other experimental hosts have.
  4. The hook command points at a path nothing creates. plugin/hooks/zcode-hooks.json:9 and the four entries after it run $HOME/.tokeneconomizer/src/plugin/scripts/te-hook. No step clones the repository there, and the README section (README.md:188-194) has no clone step at all, so following it gives a hook that fails on every event. ~/.tokeneconomizer is also the plugin's state directory. Please give the README section a git clone and say which path goes in the command, as the opencode and Hermes sections do with $PWD/TokenEconomizer/..., and use the same path in docs/ZCODE.md.
  5. Please rebase onto main: #185 and #183 changed tokeneconomizer/plugin/sidecall.py since this was opened, and it now conflicts there.

The CLA check also needs a @microsoft-github-policy-service agree reply on this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants